Several different types of pointing devices allow for pointing to selected targets on a displayed image based upon manual user input. Examples of such pointer devices include conventional keyboards and pointing devices such as a mouse or a conventional laser pointer. Most input devices require a certain degree of manual dexterity. Physically challenged users (e.g., users without hands, or with permanent or temporary loss of motor skills) may have difficulty operating keyboards and conventional pointing devices.
A user may also be challenged by virtue of simultaneously performing another task such as, for example, a doctor operating on a patient and someone driving a car or operating machinery.
During many surgical procedures, a surgeon (instructor) is accompanied by an assistant (trainee) and/or surgical residents that are also undergoing training. The procedure itself requires the instructor and the trainee to work in close collaboration and to be in constant communication. During advanced minimally invasive surgical procedures (laparoscopic or robotics-assisted), the procedure is performed through small incisions while the surgical site is observed on several video monitors. Typically, during these procedures, the instructor and assistant trainee may each have both hands occupied by surgical instruments. In addition, both may have their own operating monitor on which the surgical image appears. Until now, during instruction the instructor would have to release one of his or her instruments and point to the trainee's monitor with a finger or other mechanical pointer to provide instruction. Some have adapted a laser pointer for this purpose, allowing the instructor to keep his or her hands occupied, but this still requires that the instructor turn away from his or her monitor and point onto the trainee's monitor. This is a time consuming, disorientating, and potentially uncomfortable practice.
Known types of pointing devices that allow hands-free control of a displayed image include speech-to-text converters. Various types of head tracking technologies also exist for controlling displayed images dependent upon movement of a user's head.
Zhang et al. (“Human-computer interaction system based on nose tracking,” L. Jacko (Ed.), Human Computer Interaction, Part III, HCII 2007, LNCS 4552, pp. 769-778, 2007) and Gorodnichy et al. (Image and Vision Computing, vol. 22, pp. 931-942, 2004) introduce tracking systems that allow the user to control a pointer with movement of their nose. This type of technology uses a calibrated camera and image processing to identify the position of the nose in real-time. These are hands-free systems based on feature recognition and can be combined with vocal recognition.
Similarly, Atienza et al. (Image Processing and Communications, vol. 6, no. 3-4, pp. 47-60, 2001) describe a hands-free interface that is based on tracking facial features.
Felzer et al. (SIGACCESS newsletter, 88, pp 19-28, June 2007) present a hands-free control to assist disabled people. The hands-free pointer control system allows a pointer on a computer monitor to be controlled with a muscle-based interface consisting of a strap that is worn over the forehead.
Commercially available systems for hands-free pointer control include the SmartNav™ system. This system tracks the position of a reflective dot using infrared (IR) light. A problem with the use of IR light is that its use is susceptible to glare (J. Hargrave-Wright, “Improvements in hands-free access to computers,” Joint Information Systems Committee (JISC) Technology and Standards Watch, Report # TSW 02-06, October 2002. Available: http://www.jisc.ac.uk/media/documents/techwatch/tsw_02-06.pdf).
Other hands-free pointer control systems include the Headmouse Extreme™ and the Tracker Pro™ by Madentec™. Another system, the Boost Tracer™, uses a gyroscope to detect head motion and control a pointer on a computer monitor.
Jayaraman et al. also describe a hands-free pointer that uses a camera to track the location of a marker placed on the surgeon's mask (Scientific Session of the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES), Philadelphia, Pa., USA, Abstract #19547, P236, Apr. 9-12, 2008). This system demonstrated several difficulties including: accurate positioning of the pointer at the target site, degradation of the image quality and size, and cumbersome and complicated installation of required equipment.
The pointer systems of the prior art are insufficient in that: hand-held laser pointers systems are cumbersome and can only be used to point to one monitor at a time; there is no commercially available system for surgical training; pointer systems developed for remote collaboration require the use of a hand or a finger to actuate; most hands-free systems have been designed for disabled people, requiring movement of certain muscles or image processing to identify certain features on the user's face, which is not feasible for surgical applications since the surgeon is wearing a face mask, eye protection and a head lamp such that facial features are not exposed for tracking; systems based on gyroscopes or accelerometers have poor resolution and control at low speeds, although accuracy is critical in surgical training; systems based on infrared light are susceptible to glare or reflections occurring on other objects in the field of view of the light source leading to potentially critical errors; no system currently available combines marker tracking technology with inertial sensors; and none of the systems address the issue of velocity-dependent motion control providing precise pointer control at low speeds and large pointer displacement at high speeds.
In addition, none of the above mentioned hands-free devices have been adapted for specific situations such as during video-assisted operative procedures such as laparoscopic surgery and robotics-assisted surgery wherein the surgeon's hands are occupied manipulating surgical tools and the surgeon and trainee(s) may be using different monitors for guidance.
Therefore, what is required is a system, device and method that provide hands-free control of a cursor on a display device, such as a pointer on a monitor, that overcome the problems of the prior art. What is also required is a hands-free pointer that provides both accurate tracking and the ability to quickly move the pointer across the field of a monitor. Additionally, the hands-free pointer must not be susceptible to interference from other devices in a surgical environment, such as other lights and glare.